Search for: All records

Research-based assessments (RBAs) allow researchers and practitioners to compare student performance across different contexts and institutions. In recent years, research attention has focused on the student populations these RBAs were initially developed with because much of that research was done with “samples of convenience” that were predominantly white men. Prior research has found that the Force Concept Inventory (FCI) behaved differently for men and women using differential item functioning (DIF) analysis. We extend this research in two ways. First, we test the FCI for DIF across the intersection of gender and race for Asian, Black, Hispanic, White, and White Hispanic men and women. Second, we apply the Eaton and Willoughby five-factor model of the FCI to interpret the results of the DIF analysis. We found large DIF on a large number of FCI items. The patterns of items with large DIF follows the five-factor model. The alignment of DIF with this factor structure, along with the measurement invariance of this factor structure across these ten social identities, indicates that the items on the FCI are likely not biased but are instead measuring real differences in physics knowledge among these groups. We frame these differences as educational debts that society owes to these marginalized groups that physics instruction needs to actively repay. 

Physics instructors and education researchers use research-based assessments (RBAs) to evaluate students' preparation for physics courses. This preparation can cover a wide range of constructs including mathematics and physics content. Using separate mathematics and physics RBAs consumes course time. We are developing a new RBA for introductory mechanics as an online test using both computerized adaptive testing and cognitive diagnostic models. This design allows the adaptive RBA to assess mathematics and physics content knowledge within a single assessment. In this article, we used an evidence-centered design framework to inform the extent to which our models of skills students develop in physics courses fit the data from three mathematics RBAs. Our dataset came from the LASSO platform and includes 3,491 responses from the Calculus Concept Assessment, Calculus Concept Inventory, and Pre-calculus Concept Assessment. Our model included five skills: apply vectors, conceptual relationships, algebra, visualizations, and calculus. The "deterministic inputs, noisy 'and' gate'' (DINA) analyses demonstrated a good fit for the five skills. The classification accuracies for the skills were satisfactory. Including items from the three mathematics RBAs in the item bank for the adaptive RBA will provide a flexible assessment of these skills across mathematics and physics content areas that can adapt to instructors' needs. 

In physics education research, instructors and researchers often use research-based assessments (RBAs) to assess students’ skills and knowledge. In this paper, we support the development of a mechanics cognitive diagnostic to test and implement effective and equitable pedagogies for physics instruction. Adaptive assessments using cognitive diagnostic models provide significant advantages over fixed-length RBAs commonly used in physics education research. As part of a broader project to develop a cognitive diagnostic assessment for introductory mechanics within an evidence-centered design framework, we identified and tested the student models of four skills that cross content areas in introductory physics: apply vectors, conceptual relationships, algebra, and visualizations. We developed the student models in three steps. First, we based the model on learning objectives from instructors. Second, we coded the items on RBAs using the student models. Finally, we then tested and refined this coding using a common cognitive diagnostic model, the deterministic inputs, noisy “and” gate model. The data included 19 889 students who completed either the Force Concept Inventory, Force and Motion Conceptual Evaluation, or Energy and Momentum Conceptual Survey on the LASSO platform. The results indicated a good to adequate fit for the student models with high accuracies for classifying students with many of the skills. The items from these three RBAs do not cover all of the skills in enough detail, however, they will form a useful initial item bank for the development of the mechanics cognitive diagnostic. 

This investigation examines the efficacy of multilevel analysis of individual heterogeneity and discriminatory accuracy (MAIHDA) over fixed-effects models when performing intersectional studies. The research questions are as follows: (1) What are typical strata representation rates and outcomes on physics research-based assessments? (2) To what extent do MAIHDA models create more accurate predicted strata outcomes than fixed-effects models? and (3) To what extent do MAIHDA models allow the modeling of smaller strata sample sizes? We simulated 3,000 data sets based on real-world data from 5,955 students on the LASSO platform. We found that MAIHDA created more accurate and precise predictions than fixed-effects models. We also found that using MAIHDA could allow researchers to disaggregate their data further, creating smaller group sample sizes while maintaining more accurate findings than fixed-effects models. We recommend using MAIHDA over fixed-effects models for intersectional investigations. 

Abstract Culturally relevant pedagogy (CRP) seeks to improve equity in instruction and leverage students’ experiences by promoting academic success, cultural competence, and sociopolitical consciousness. We examine instructors’ perceptions of student identity to understand the ways undergraduate mathematics instructors are enacting or experiencing barriers to enacting CRP. Interviews with ten mathematics faculty at Hispanic-serving institutions identified two potential barriers to enacting CRP: first, instructors’ hesitance to communicate about student identity, especially with respect to race and gender; and second, instructors holding epistemologies that mathematics is culture-free. Despite these barriers, almost all interviewees implemented the academic success tenet of CRP. These barriers may prevent instruction around cultural competence and sociopolitical consciousness, which are the two tenets that most capitalize on students’ informal knowledge, identities, and cultural experiences. Changing discourse by taking more risks in conversation and inviting a more diverse range of people to the undergraduate mathematics community are potential ways to address these barriers. 

Abstract Background The growing understanding of the oppressive inequities that exist in postsecondary education has led to an increasing need for culturally relevant pedagogy. Researchers have found evidence that beliefs about the nature of knowledge predict pedagogical practices. Culturally relevant pedagogy supports students in ways that leverage students’ own cultures through three tenets: academic success, cultural competence, and sociopolitical consciousness. If STEM practitioners believe that their disciplines are culture-free, they may not enact culturally relevant pedagogy in their courses. We investigated how and in what forms 40 faculty from mathematics, physics, chemistry, and biology departments at Hispanic-Serving Institutions enacted culturally relevant pedagogy. We used the framework of practical rationality to understand how epistemological beliefs about the nature of their discipline combined with their institutional context impacted instructors’ decision to enact practices aligning with the three tenets of culturally relevant pedagogy. Results In total, 35 instructors reported using practices that aligned with the academic success tenet, nine instructors with the cultural competence tenet, and one instructor with the sociopolitical consciousness tenet. Instructors expressed and even lauded their disciplines’ separation from culture while simultaneously expressing instructional decisions that aligned with culturally relevant pedagogy. Though never asked directly, six instructors made statements reflecting a “culture-free” belief about knowledge in their discipline such as “To me, mathematics has no color.” Five of those instructors also described altering their teaching in ways that aligned with the academic success tenet. The framework of practical rationality helped explain how the instructors’ individual obligation (to the needs of individual students) and interpersonal obligation (to the social environment of the classroom) played a role in those decisions. Conclusions Instructors’ ability to express two contradictory views may indicate that professional development does not have to change an instructor’s epistemological beliefs about their discipline to convince them of the value of enacting culturally relevant pedagogy. We propose departmental changes that could enable instructors to decide to cultivate students’ cultural competence and sociopolitical consciousness. Our findings highlight the need for future research investigating the impacts of culturally relevant pedagogical content knowledge on students’ experiences. 

Researchers often frame quantitative research as objective, but every step in data collection and analysis can bias findings in often unexamined ways. In this investigation, we examined how the process of selecting variables to include in regression models (model specification) can bias findings about inequities in science and math student outcomes. We identified the four most commonly used methods for model specification in discipline-based education research about equity: a priori, statistical significance, variance explained, and information criterion. Using a quantitative critical perspective that blends statistical theory with critical theory, we reanalyzed the data from a prior publication (Van Dusen & Nissen, 2020) using each of the four methods and compared the findings from each. We concluded that using information criterion produced models that best aligned with our quantitative critical perspective’s emphasis on intersectionality and models with more accurate coefficients and uncertainties. Based on these findings, we recommend researchers use information criterion for specifying models about inequities in STEM student outcomes.